Software testing and automating software testing is one of the key topics in software engineering. Typical sets of activities performed in software testing are test plan creation related to the objective, identifying test cases, executing test cases against the software, and verification and certifying the software based on execution results.
Types or classes of testing may be listed as business level system/acceptance testing, integration level testing for checking co-existence with other parts, unit testing for certifying basic units of development, and technical or performance testing to verify the stability and the loading characteristics. Many tools are available today to perform these activities in specific domains.
Issues faced by the software testing groups are ensuring that the set of specifications available to testers are reliable with respect to implementation, identifying all the paths through the software flow to create test cases, identification of support drivers to aid in testing the units independently, and identification of data requirements and creating an execution plan. These issues faced are primarily due to the lack of specifications that are structured and reliable to enable test planners to come up with a comprehensive plan.
Moreover, when changes are made to the software, the relation to the cases that need to be tested is created manually and is prone to oversights or increase in the testing load when over cautious. Another key problem is, since the testing group may be part of the development, the issue of knowledge creation needed to write valid test plans is a major issue. In most of the projects, a lot of resources are consumed in communicating expected behavior to the testing group.
Software developed for many applications are tested with provision for manual entry and logging of these test cases in documents. These documents need to have naming convention in the test cases. There needs to be a sequence to execute them and any software issue arising out of this testing is also manually communicated. Files that have this data are retained as soft copies and updated in a secure storage system; otherwise they are printed as hard copies and retained for reference. An ability to streamline this testing process and collect the both sequence and results automatically enabling a robust software release with proper testing completion is not accomplished without manual support.
A test plan generated for a conventional development process is different from the test plan that used for a maintenance or enhancement release. This test plan for an enhancement release needs analysis of the existing test case and execution dependencies that will provide a sufficient and complete list of test cases in the test plan for execution.
Testing of any developed software can be done in a number of ways. Streamlining the testing process having a well documented schedule for the test, prior to its initiation is a major overhead for most organizations. The awareness that generally testing takes more time than development justifies the complexity behind the process. To ensure an integrated environment is provided for users to record their testing sequences, automating the process of testing and also provide estimate for the testing to be done is a major hurdle.
It is very hard to formally identify all the paths that need to be tested. The paths also need to be classified as business system level cases, integration cases and Unit cases based on the staged testing approach. For technical testing there is a need to create volume testing plans to validate all the hot spots in the implementation.
List of issues that need to be addressed are representation of software specifications in a structured format that is understood by the testing group, creating an ability to view and pick the various paths through the software structure, creating an ability to classify and record cases as part of a plan, and support for creating execution of test cases and the results.